Method and system for measuring quality of performance and/or compliance with protocol of a clinical study

ABSTRACT

A method is proposed for determining clinical study compliance. The method includes obtaining criteria for the clinical study and accessing stored clinical data relating to the clinical study. Thereafter, the criteria and clinical data are correlated to determine a measure of compliance with the criteria of the clinical study.

The present application hereby claims priority under 35 U.S.C. §119 onU.S. provisional patent application No. 60/545,164 filed Feb. 18, 2004,the entire contents of which are hereby incorporated herein byreference.

FIELD OF THE INVENTION

The present invention is generally related to the field of clinicalstudies.

BACKGROUND OF THE INVENTION

The framework for traditional business models for clinical studies hasbeen rather stable over the last few decades. In such a business model,a sponsor (such as a pharmaceutical company which has developed a newdrug, for example) paid all participants which performed in the study.At a minimum, these included participating patients and a medical doctor(an investigator) in charge of supervising the patients. In many cases,an investigation or clinical trial site (e.g., a hospital) wasadditionally included, where one or more investigators was employed.

So-called contract research organizations (CROs) further establishedtheir services in the workflow chain of clinical studies, in between thesponsor on one end, and the investigator and patients on the other. TheCRO often took over the complete management of the clinical study,including all necessary services including, for example, development ofstudy protocol, recruiting patients and investigators and/orinvestigation sites, contracting the participants, supervising theconductance of the study, collecting and evaluating data, channeling thepayment from the sponsor to the participants, etc. Of course, for suchservices, the CRO received a substantial part of the aforementionedpayment for their own services.

When recruiting the patients, the CRO, or even the sponsor, tended touse and still uses crude methods wherein prospective patients fill outforms and are screened as candidates for clinical studies. The datautilized is normally that obtained from the patient himself or herself.Regarding the investigator or investigator/clinical trial site chosen toconduct/monitor/etc. the study, information previously obtained by thesponsor or CRO was typically used. However, this was often a slowprocess which often did not produce an ideal patient, investigatorand/or investigator/clinical trial site.

FIG. 1 illustrates a typical traditional system for use in connectionwith clinical studies. Initially, a sponsor 100 (such as a drugmanufacturer, for example) defines the study requirements or criteria(study parameters, study protocol, etc.) for the particular clinicalstudy in question. A CRO 120 may then be employed to manage the study,noting that the CRO 120 may develop the study requirements or criteriaof the clinical study or may assist therein. The CRO may also assist inrecruiting patients for the study, as well as selecting an appropriateinvestigator/investigators and appropriate clinical trial site(s). If aCRO is involved, the CRO is paid by the sponsor 100. The CRO thenmanages the study and then pays others involved in the study includinginvestigators 130, patients 140, and potentially investigation orclinical trial sites such as hospitals, for example (not shown).

In addition, the quality of the data acquired must be maintained forproper results in clinical trials, and for the ultimate approval of anew drug, for example. Quality of clinical raw data such as medicalimages or physiological data can to some extent be checkedretrospectively (see, for example German application no. DE 10 2004 008197.2 entitled “Verfahren zur Qualitätskontrolle von je anunterschiedlichen, aber vergleichbaren Patientenkollektiven im Rahmeneines medizinischen Vorhabens erhobenen medizinischen Datensätzen”,filed Feb. 9, 2004, the entire contents of which are incorporated hereinby reference). However, the quality of a database acquired during aclinical study does not only depend on the inherent quality of the rawdata (such as signal noise, the wrong use of the measurement device,miscalibration of the device, etc.), but also depends considerably onthe workflow context in which the data had been generated. The “standardoperating procedure” (SOP) of a clinical trial also regulates manydetails of the workflow context in which the patient participating inthe clinical study has to be treated. This is part of the qualityregulations of a clinical trial.

As previously indicated, the requirements for performing a clinicalstudy may be defined in a study protocol, which may be defined by thesponsor 100 and/or the CRO 120. Although participants in the clinicaltrial confirm, by signing a contract for example, that they are going tocomply with the quality regulations, there is no systematic and/orautomated way to check this quality compliance. Due to the inability tomeasure quality compliance in clinical workflow, the payment to thepatients, investigators, CRO, investigator/clinical trial sites, etc.does not depend on quality of data.

SUMMARY OF THE INVENTION

The present inventors have recognized problems with the traditionalclinical study model, and an object of the present application is toimprove on the traditional clinical study model, and thus improve theclinical study or clinical study process. The present inventors havefurther recognized a need for objective, and preferably automated,supervision of compliance with quality criteria and/or a need for asystematic and preferably automated way to check quality compliance of aclinical study.

The present inventors have further recognized that the advent ofinformation technology (IT)—based clinical workflow systems and datamanagement systems, such as the products “SOARIAN” or “SYNGO” fromSiemens AT medical solutions for example, provide some of the toolswhich can be used in providing an automated check of quality regulationsduring clinical workflow in the context of a clinical trial for example.Data management systems like “SYNGO” store and manage not only clinicaldata and images, but to some extinct, the context in which the data havebeen generated. For example, in the standardized DICOM header of medicalimages, facts like the use of contrast agents, for example, areincluded. The image modality systems, which connect to “SYNGO” and“SORIAN”, safe procedures associated with the imaging processes. This,as recognized by the present inventors, permits a new and novelretrospective analysis of the processes as set forth in one embodimentof the present application.

In a similar way, workflow systems like “SORIAN”, for example,distribute sub-tasks of clinical workflow to responsible persons,schedule and optimize sequence of actions in a clinical workflow, etc.Thus, as recognized by the inventors, they can be used to automaticallyreport aspects such as, for example, time, place, responsible personsthat performed actions, use devices and materials, etc. The presentinventors have recognized that this information can be used, in a uniqueway, to improve quality compliance of a clinical study, and even toimprove quality compliance in any type of clinical workplace.

One specific object involves the use of clinical IT infrastructure toderive, when correlated with obtained criteria of a clinical study, ameasure of compliance with the clinical study. Further, the criteria forthe clinical study may include at least one of patient and non-patientcompliance criteria and/or compliance/non-compliance of a clinical trialsite. The present inventors have recognized these and other needs forimproving a clinical study compliance.

Further, the inventors have recognized that in the traditional settingof a clinical study, the CROs and sponsors 100 had no access to this ITinfrastructure. The investigation or clinical trial sites such as ahospital, for example, were the owner of such IT infrastructure anddatabases. However, as these investigation/clinical trial sites werebiased parties and thus sponsors 100 of clinical studies and CROs 120were not interested in their involvement to the extent of using their ITinfrastructure.

The present inventors, in one embodiment of the present invention, haverecognized that further value of both existing clinical IT databasesstoring clinical data, as well as IT-based clinical workflow managementsystems and data management systems developed by the present assignee,and have thus developed further uses for such systems in the area ofclinical study compliance; and even in the area of hospital procedurecompliance.

In one embodiment, a method of determining clinical study compliance ofthe present invention includes receiving (and/or even storing) criteriafor the clinical study; accessing (or even storing) clinical datarelating to the clinical study; and correlating the criteria andclinical data to determine a measure of compliance with the criteria ofthe clinical study. The method may further include calculating a measureof compliance for a plurality of criteria for the clinical study.

The measure of compliance can then be used in at least one of, forexample, calculating a payment based upon the determined measure ofcompliance (by a clinical trial site, for example); calculating aranking (for multiple clinical trial sites, for example); and/ordetecting and/or reporting quality problems (during the course ofconducting a clinical study, for example). In addition, weightedcriteria may be used. This weighted criteria can include weighting thataffects the measure of compliance, for example.

In another embodiment, the invention relates to a method of monitoring aclinical study at a trial site, including accessing (or even storing) atleast one protocol for the clinical study; accessing (or even storing)clinical data relating to actions performed at the clinical trial sitebased upon the at least one protocol; and correlating the at least oneprotocol and clinical data and using the correlated information tomonitor the clinical study at the clinical trial site. Such a method canfurther include determining a measure of compliance of clinical studyprotocol at the clinical trial site, based upon the correlation, whereinthe clinical trial site may be a hospital, for example.

In yet another embodiment, the invention relates to a method ofimproving compliance within at least one clinical study at a clinicaltrial site. The method includes obtaining at least one protocol for theclinical study; and planning a plurality of clinical actions incompliance with criteria set forth in the at least one protocol, whereinthe planning is performed at least partially automatically. The criteriamay include at least one of rules and constraints, and the at least oneof rules and constraints may be stored in machine readable form.Further, the planning may include at least one of scheduling use ofequipment at the clinical trial site and scheduling a clinical actionitem at the clinical trial site.

The present inventors, in one embodiment of the present invention, alsorecognized the importance of the introduction of some type of qualitycontrol and benchmarking measures. By inclusion of various measures, thepayment for the clinical study can be made to be performance/outcomeoriented, rather than oriented as contracts for upfront fixed amounts.

Other embodiments of the present application may include devices forimplementing any of the aforementioned methods, programs adapted toperform any of the aforementioned methods when executed on a computerdevice, and/or computer readable mediums storing any of theaforementioned programs.

Additional embodiments of the present application may include systemsfor implementing any of the aforementioned methods.

For a full understanding of the nature and advantages of the variousaspects of the invention, reference should be made to the detaileddescription of exemplary embodiments taken in conjunction with theaccompany drawings. The detailed description provides only exemplaryembodiments of the invention and thus, the claims of the presentinvention should not be limited as such.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will become more fully understood from thedetailed description of preferred exemplary embodiments givenhereinbelow and the accompanying drawings, which are given by way ofillustration only and are thus not limitive of the present invention,and wherein:

FIG. 1 illustrates a typical traditional model for use in clinicalstudies;

FIG. 2 is an example of an aspect of an embodiment of the presentapplication illustrating the server and its connection to variousdatabases;

FIG. 3 includes an exemplary embodiment of the overall process ofvarious aspects of the present application;

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS OF THE PRESENTAPPLICATION

In one embodiment, the present invention is directed to a method ofdetermining clinical study compliance. The method includes obtainingcriteria for the clinical study, and then accessing stored clinical datarelating to the clinical study. Thereafter, the criteria and clinicaldata are correlated to determine a measure of compliance with thecriteria of the clinical study. In a further embodiment, a measure ofcompliance for a plurality of criteria for the clinical study arecalculated. The method can be performed utilizing clinical informationtechnology (IT) infrastructure and then, if needed, additional databasescan be created.

Clinical data can include data stored in a database of existing clinicalIT infrastructure, such as an electronic healthcare database, forexample. This can include, but is not limited to at least one of adatabase with electronic patient records, a database of clinicalworkflow management system, information from a hospital IT system(financial or clinical), information from a laboratory or radiologyinformation system, information from a picture archiving andcommunication system (PACS), information from a physician's IT system,for example, etc.

In an embodiment of the present application, use is made of clinical ITinfrastructure and databases, such as electronic patient records (EPR),hospital information systems (HIS) and/or clinical workflow managementsystems. In an embodiment of the present application, such clinical ITinfrastructure and databases, storing various types of clinical data,are correlated with obtained criteria for the clinical study, todetermine a measure of compliance with the criteria of the clinicalstudy. More specifically, in one embodiment, the system/method of thepresent application uses this correlation to check on how aninvestigator/clinical trial site/patient/etc. is complying with variousparameters or requirements of a clinical study.

In another embodiment, the present invention is directed to a method ofmonitoring a clinical study at a clinical trial site. As such, a sponsorof a clinical study can check or monitor how aninvestigator/patient/clinical trial site, is complying with requirementsand/or other criteria of a study. The method can include obtaining atleast one protocol for the clinical study; accessing stored clinicaldata relating to actions performed at the clinical trial site based uponthe at least one protocol; and correlating the at least one protocol andclinical data and using the correlated information to monitor theclinical study at the clinical trial site. The method, in oneembodiment, can further include determining a measure of compliance ofclinical study protocol at the clinical trial site, based upon thecorrelation.

As shown in FIG. 2 of the present application, an analytical device 200has been developed. This analytical device 200 can be a type of computerdevice/processor and/or server which is networked or otherwise hasaccess to clinical IT infrastructure and which is further networked to,or can otherwise receive criteria regarding a clinical study and receivedata with patient identification (e.g. patients name and birthday, or apatient identification code such as a patient social security number),wherefrom a Patient's ID Database is built.

The analytical device 200 then may derive rules from the criteria. Theserules may include, but are not limited to, rules which influence theplanning of a clinical action, rules which may be applied to checkresults from a clinical action for their compliance with the criteria,etc. These rules may be converted (if necessary) by the analyticaldevice 200 into a machine-readable form, which can then be interfaced toand understood by, for example, the clinical workflow management system.Then, the rules may be applied, for example, to influence the planningof clinical actions for all patients with Ids contained in the patientID data base; may be applied to check results of clinical actionsinvolving these patients for there compliance with the criteria; etc.

The analytical device 200 is able to access and analyze clinical data,such as that stored in any of the clinical workflow management system210, EPR 212, HIS 214 (or any other type of clinical IT infrastructureand/or database). This analytical device 200 connects or is otherwisenetworked to, and can thereby access/receive/obtain and then analyzedata from any of the clinical workflow management system 210, EPR 212,HIS 214 (or any other type of clinical IT infrastructure and/ordatabase). The analytical device 200 may further be networked orotherwise connected to the sponsor 220, the clinical study SOP database280 and/or the CRO 230. The analytical device 200 can then receive orotherwise obtain criteria for a clinical study from the sponsor 220,clinical study SOP database 280 and/or CRO 230 and can then analyze theobtained clinical data in conjunction with (or based upon) the obtainedcriteria for a clinical study. The analytical device 200 of anembodiment of the present application is then able to correlate theclinical data in conjunction with (or based upon) the obtained criteriafor a clinical study, to determine a measure of compliance with thecriteria of the clinical study. Alternatively or in addition thereto,the analytical device 200 is able to use the correlated information tomonitor the clinical study at the clinical trial site.

As shown in FIG. 2, the analytical device 200 can receive requirementsof the clinical study directly from the sponsor 220, which can includethe criteria for the clinical study; and/or can receive such informationfrom the CRO 230 managing the study; noting that the CRO 230 may take onall necessary services for managing the study including, but not limitedto development of a study protocol, recruiting patients andinvestigators and/or investigation or trial sites, contracting theparticipants, supervising the conductance of the study, collecting andevaluating the data and channeling the data from the sponsor to theparticipants. Thus, the CRO 230 and/or sponsor 200 may transmitinformation regarding desired/necessary criteria of the clinical study(and even desired target parameters) to the analytical device 200.Accordingly, the analytical device 200, in some way, obtains access tothe information in the clinical study SOP database 280.

Clinical data of a plurality of clinical trial or investigationsites/patients/investigators/etc., and the obtained criteria, may befurther analyzed to determine clinical trial or investigation siteswhich meet or exceed target or thresholds of compliance with theobtained criteria and such plural patients/clinical trialsites/investigators/etc may be ranked accordingly. This rankedinformation can then be output or otherwise sent to the sponsor 220and/or CRO 230 for use in determining desired patients/clinical trialsites/investigators/etc. for continued use and/or termination regardingthe clinical study.

“Criteria”, as referenced throughout the embodiments of the application,refers to clinical study criteria. These “criteria” are importantaspects of the clinical study. These criteria of the study can be usedby the analytical device 200. Thus, the criteria outline key or otherimportant aspects of the study which, when provided and correlated withclinical data, can help produce measures of compliance of the clinicalstudy that can be used, for example, to monitor the clinical study atthe clinical trial site.

Some non-limiting examples of “criteria” may include, but are notlimited to e.g.:

-   -   Number of patients needed for the clinical study;    -   Patient inclusion criteria such as, for example, patients with a        given diagnosis (e.g., diabetes type I, for example). Another        example of patient inclusion criteria can be, for example, age        group (e.g., 40-60 years) of patients to be included in the        study;;    -   Patient exclusion criteria: Patients not previously diagnosed        with an ailment, (hypertension, for example). Another example of        patient exclusion criteria can be, for example, patients not        having been prescribed with a given medication “x” previously;    -   Accompanying exams to be undertaken during the study (aside from        prescribing the medication under study) to the patient: This can        include, but is not limited to regular (i.e., weekly, daily,        etc.) control of blood pressure, heart rate, etc.; Making        diagnostic images for the therapy success control every “x”        days/months/years;    -   The drug under study has to be taken by the patient 3 times a        day, always before the meals (breakfast, lunch, dinner) (med.:        pre-prandial medication); etc.

Often, elements relating to these “criteria” cannot be measureddirectly, but must be deduced from other measurable parameters orclinical data, and perhaps from a combination of other measurableparameters or other measurable clinical data. This may include volatiledata which may be entered and/or stored for only a short time, and laterdeleted or erased. For example, the exact time as to when a patient hastaken a drug or eaten his meal is usually not measured and recorded by anurse. However, this time is typically recorded in the Clinical WorkflowManagement System when the drug and the meal are scheduled for deliveryto the patient. Therefore, the criterion “pre-prandial medication” canat least, to some certainty, be indirectly deduced from the two databaseentries including “Scheduled time for drug delivery to patient” and“Scheduled time for meal delivery to patient”, provided that the patienthas taken medication and is eating meals immediately after delivery. Assuch, an attempt may be made to correlate volatile clinical data withcriteria for the clinical study. If so, the volatile data may bepermanently stored upon determining a lack of correlation. Also,volatile clinical data may be correlated with criteria for the clinicalstudy, and the volatile data may be permanently stored upon determiningthat the measure of compliance is below a threshold. Further, clinicaldata may be stored permanently, upon determining that the data relate toa patient enrolled in the clinical study.

Thus, the analytical device 200 can, for example, build an empiricaldatabase for use in such situations, which contains rules on how tocombine measurable indirect criteria in order to derive from these, aprobability that a non-measurable criterion is met. Accordingly, theanalytical device 200 can create a type of mathematical formula orweighting factors regarding the combining of several direct and indirectaspects of the criteria into a weighted combination. Most likely, thisformula will include a weighted sum or weighted product of a singlecriteria. This can then be correlated with existing clinical data fromthe clinical IT infrastructure to derive a measure of compliance and/orto monitor a clinical study at a clinical trial site, for example.Thereafter, an output ranking may be derived from the calculated/derivedperformance measures, the ranking being based upon weighteddeterminations using the weighing factors. Weighted criteria may beused. This weighted criteria can include weighting that affects themeasure of compliance, for example. Further, the ranking can be for anyor all of single or multiple patients/clinical trialsites/investigators/etc.

The assignee of the present application has further been involved invarious other inventions regarding clinical studies, and in some casesthe use of clinical IT infrastructure, in order to improve thedevelopment of clinical study monitoring and/or compliance; thedevelopment of clinical study protocols; improving the effectiveness ofpatient recruiting; controlling the compliance of clinical studyprotocol rules; etc. The entire contents of each of the followingapplications is hereby incorporated by reference in the presentapplication:

-   -   “Procedure to Identify Eligible Study Patients in an All-Day        Setting” (U.S. provisional application Ser. No. 60/545,169,        filed Feb. 18, 2004) and corresponding U.S. non-provisional        application entitled “A Method Of Recruiting Patients For A        Clinical Study”, assigned U.S. application Ser. No. ______, and        filed on Oct. 28, 2004;    -   “Incentive-System for Clinical Trials” (U.S. provisional        application Ser. No. 60/545,170, filed Feb. 18, 2004), and        corresponding U.S. non-provisional application entitled “A        Method Of Monitoring Patient Participation In A Clinical Study”,        assigned U.S. application Ser. No. ______, and filed on Oct. 28,        2004;    -   “Procedure Providing a Benchmarking of Clinical Test Sites and a        Concomitant Method of Quality-Based Monetary Compensation”;        (U.S. provisional application Ser. No. 60/545,165, filed Feb.        18, 2004) and corresponding U.S. non-provisional application        entitled “A Method Of Examining A Plurality Of Sites for A        Clinical Trial”, assigned U.S. application Ser. No. ______, and        filed on Oct. 28, 2004;    -   “Risk-Sharing Business Model for the Use of HIS Data to Improve        Cost Effectiveness of Clinical Studies” (U.S. provisional        application Ser. No. 60/545,168, filed Feb. 18, 2004) and        corresponding U.S. non-provisional application entitled “A        Method Of Improving A Clinical Study”, assigned U.S. application        Ser. No. ______, and filed on Oct. 28, 2004;    -   “Quality Compliance Improvement in Clinical Studies using        IT-Based Clinical Workflow Systems” (U.S. provisional        application Ser. No. 60/545,164, filed Feb. 18, 2004) and        corresponding U.S. non-provisional application entitled “Method        and System For Measuring Quality of Performance and/or        Compliance with Protocol of a Clinical Study”, assigned U.S.        application Ser. No. ______, and filed on Oct. 28, 2004;    -   Verfahren zur Durchführung einer klinischen Studie (DE 10 2004        008 196.4);    -   Verfahren zur Überprüfung der Durchführbarkeit eines        medizinischen Vorhabens mit Aufnahmekriterien für Patienten (DE        10 2004 008 189.1);    -   Verfahren zur Qualitätskontrolle von je an unterschiedlichen,        aber vergleichbaren Patientenkollektiven im Rabmen eines        medizinischen Vorhabens erhobenen medizinischen Datensätzen (DE        10 2004 008 197.2);    -   Verfahren and Einrichtung zur Überprüfung der Einhaltung einer        Durchführungsvorschrift fur eine an einem Patienten        durchgeführte medizinische Maβnahme (DE 10 2004 008 190.5);    -   Verfahren zur Qualitätsbewertung von elektronisch gespeicherten,        insbesondere medizinischen, Wissensdaten (DE 10 2004 008 191.3);    -   Verfahren zur Auswahl eines moglichen Teilnehmers für ein        medizinisches Vorhaben anhand eines Auswahlkriteriums (DE 10        2004 008 192.1);    -   Verfahren and Informationssystem zur Durchführung einer        klinischen Studie an einem Patienten. (DE 10 2004 008 194.8);    -   Verfahren zur Überprüfung der Einhaltung einer einem        medizinischen Arbeitsablauf zugeordneten Durchführungsvorschrift        (DE 10 2004 008 195.6); and    -   Verfahren zur Auswahl eines Teilnehmers für ein medizinisches

Vorhaben mit Auswahlkriterien für Patienten (DE 10 2004 008 188.3).

In one embodiment of the present invention. further value of suchclinical IT databases has been realized, wherein clinical data from aplurality of different investigation sites is used, especially differentinvestigation sites participating in the same clinical study. Thisinformation adds to the value that can be provided by the Hospital ITSolution Service Provider, who can develop, sell, install and maintainclinical IT solutions and databases, and in many cases can also storeand maintain related databases obtained from a correlation of thetraditional clinical IT databases and clinical study criteria.

Thus, it should be understood that FIG. 2, and each of the figures andembodiments of the present application, represents the Hospital ITSolution Provider (for manual data access) or the analytical device 200(for automated data access) with access to the clinical workflowmanagement system 210, an EPR 212 and/or an HIS 214 of one, or of aplurality of clinical trial sites. Thus, the clinical data can includedata from a plurality of clinical trial sites, and further can includedata from a plurality of previously conducted clinical trials. Clinicaldata from a plurality of clinical trial sites is thereby preferablyfurther analyzed in conjunction with the obtained criteria for aclinical study, to determine clinical trial sites which may provideprospective services (for planning, etc.) and to monitor compliance withcriteria of an existing study in a more retrospective manner. The namesof clinical trial sites can then be provided to the sponsor 220 and/orcan be ranked accordingly.

Further, the criteria for the clinical study can include both measurableparameters and parameters of context. Some examples of measurableparameters can include, but are not limited to : number of times acertain exam has been carried out, maximum blood pressure of patientduring one day, number of calories of daily diet for a patient, etc.Thus, they are things that can be measured. “Parameters of context”include parameters of context within a clinical workflow chain orclinical action item. For example, a “parameter” like “breakfast forpatient X is scheduled for 8:00 am on August 10” is probably notmeaningful by itself But if an MR exam is scheduled at 7 am, whichnecessitates fasting of the patient before the exam (med.: prae-prandialexam), then the breakfast scheduled at 8 am would be “compliant” in thiscontext. Accordingly, a breakfast scheduled at 6 pm would benon-compliant, and an according MR exam under these circumstances be ofmuch lower or no value.

An advantage that the analytical device 200 can offer, is access toclinical data such as patient data, clinical workflow data, etc., muchearlier than the CRO 230, who receives only bundled data in the form,typically, of milestone reports. An analytical device 200 has access tothe relevant clinical data in real time, and can extract and update allinformation on study-relevant clinical data such as patient data, on adaily basis for example. To this end, new software modules can also beincorporated by the Hospital IT Solution Provider into a clinicalworkflow management system 210, new data entries in the EPR database212, etc., in order to specifically collect information on a clinicalstudy, and/or to support or enable the analytical device 200 to do suchdata collection automatically. With the use of such real-time data, amuch more effective monitoring of the clinical study compliance and theachievement of clinical study milestones is possible.

FIG. 3 provides an illustration of various detailed aspects ofembodiments of a method of determining clinical study compliance, and amethod of monitoring a clinical study of the clinical trial site.Initially, the analytical device 200 (not shown) obtains criteria forthe clinical study, which may include at least one protocol for theclinical study for example. This can be achieved by accessing theclinical study “standard operating procedure” (SOP) information from adatabase 280, for example. The criteria for the clinical study may betranslated into rules which are machine-readable by the clinicalworkflow management system 210 for example and/or other aspects of theanalytical device 200, wherein applicable criteria may be selected fromthe database 200 using a graphical user interface (GUI) 285 for exampleAlternatively, the criteria for the clinical study may be automaticallytranslated into rules which are machine-readable by the clinicalworkflow management system 210 for example, or by other aspects of theanalytical device 200 including at least one of a look-up table, athesaurus, an ontology, etc. Further, the criteria for the clinicalstudy may be translated into rules which are machine-readable, whereinapplicable criteria are selected from a database.

Thus, in monitoring our clinical study for aspects or measures ofquality and/or compliance with various protocol rules of the clinicalstudy criteria, parameters of the clinical study criteria may beexamined. These can include, but are not limited to, abilities of aninvestigator and/or investigation site to perform certain tasks, thesuitability of clinical processes and the context of clinical rules orparameters of a clinical study, etc. It should be noted that“compliance” as used throughout the present application refers to ameasure of compliance with various criteria including rules, protocol,etc. and in some instances, it can be a measure of “quality” (qualitycriteria, quality improvement, quality assurance, etc.) of performancewhich is essentially monitored.

From the clinical study SOP 280 as shown in FIG. 3, a list ofwork-oriented quality regulations may be extracted. These qualityregulations, once extracted, can then be formed into a new database forquality-rated rules for clinical workflow known as the SOP Quality DataBase 300 (SOP Q-Base 300), as shown in FIG. 3. Some examples ofworkflow-flow oriented quality regulations can include, but are notlimited to:

-   -   patient examinations with examination Z being done at least 5        times per week, but maximally 7 times per week;    -   a restriction that chemicals used in investigation X are not        older than 3 months;    -   a restriction that bio-chips used within device Y include a        serial number higher than XXX; etc. Such a SOP Q-Base is        accessible by analytical device 200 and thus can be        interconnected, in some manor, with clinical workflow management        system 210, EPR 212, HIS 214, etc.

In connection with the various embodiments of the present application,different types of validation of the quality criteria of a clinicalstudy, including but not limited to determining measures of compliancewith criteria of a clinical study and monitoring other aspects of theclinical study and the clinical trial site, can include both prospectiveand retrospective quality checks. In a retrospective quality check,clinical data, including but not limited to clinical actions forexample, relating to a clinical study, may be correlated with criteriaof the clinical study to determine a type of quality check and/or ameasure of compliance with the criteria of the clinical study. In aprospective embodiment of the present application, at least one protocolfor the clinical study may be obtained, and a plurality of clinicalactions in compliance with the criteria set forth in the at least 1protocol may be planned, wherein the planning is performed at leastpartially automatically. Both aspects or embodiments of the presentapplication will be discussed hereafter in conjunction with FIG. 3.

For both modes of operation, both retrospective and prospective, oncethe criteria for the clinical study is obtained from database 280, theSOP Q-Base 300 may be established in a manor previously set forth.Further, a look-up table of analytical device 200, for example, may beused to link required information regarding patients enrolled in theclinical study and may be able to obtain this information from existingdatabases including but not limited to HIS 214 at a clinicalinvestigation site. To this end, a list of enrolled patient IDs,regarding patients enrolled for a particular clinical study, may beattached or entered into the related SOP Q-Base as shown in element 310of FIG. 3. Alternatively, a SOP Q-Base ID may be entered into thepatient's EPR database 212, for example, for identify a clinical studySOP belonging to the clinical study in which the patient isparticipating.

Elements 320 and 325 deal with the prospective and retrospective aspectsof the present application. As shown in element 320, if a patient isenrolled in the SOP Q-Base 300 for example, clinical workflow planningcan be done in accordance with at least one of criteria directlyobtained from the clinical study SOP 280 and/or rules for clinicalworkflow established in the SOP Q-Base 300. In such an embodiment of thepresent application, a method for improving compliance with at least oneclinical study of the clinical trial site can be achieved. The methodcan include obtaining at least one protocol for the clinical study fromthe clinical study SOP database 280 for example, including but notlimited to rules for clinical workflow established by the SOP Q-Base300. Once obtained, a plurality of clinical actions can then be plannedin compliance with the criteria set forth in the at least one protocol,wherein the planning may be performed at least partially automatically.

More preferably, the SOP Q-base 300 may be automatically contacted bythe clinical workflow management system 210. A clinical action may beplanned, scheduled, required, etc. by the analytical device 200 inconjunction with the clinical workflow management system 210 forexample. For example, the method of at least one of planning andscheduling a clinical action can include obtaining criteria for aclinical study and for at least one clinical action associated with theclinical study and accessing stored clinical data relating to theclinical study. Thereafter, the criteria and clinical data may becorrelated to at least one of plan and schedule the clinical action.

The analytical device 200 can then automatically take into account anyrestriction defined in the SOP Q-Base 300 or in at least one protocol ofa clinical study from database 280, wherein this can be done forparticular clinical study participants. Accordingly, it can then modifythe required clinical action. Optionally, any modification of clinicalworkflow caused by SOP restriction may be documented such that it canlater be justified as to why this clinical workflow situation deviatedfrom normal procedures. This can be done by documenting such informationin an electronic patient record (EPR) 212 together with an SOP ID (name,etc.), for example.

The criteria of the at least one protocol of the at least one clinicalstudy may include rules and/or constraints. The rules and/or constraintsmay be stored in machine readable form, such as in the SOP Q-Base forexample. The planning may include, but is not limited to, at least oneof scheduling use of equipment at the clinical site, use of personnel atthe clinical trial site, scheduling a certain sequence of subsequentclinical actions, etc. Thus, planning in a clinical trial site, such asa hospital, can take place at least partially automatically, includingat least one of the scheduling of use of equipment, the planning of useof personnel, and scheduling a certain sequence of subsequent clinicalactions, etc. Further, the at least one of planning and scheduling caninclude at least one of planning and scheduling at least one subsequentclinical action.

In general, a clinical workflow management system coordinates clinicalservices across team members and institutions. It permits the planning,execution, and tracking of clinical actions, and documents and reportsthe clinical actions performed. For planning and scheduling actions, aworkflow engine, a rules engine and definitions database are used toidentify the relevant sets of tasks or to identify workflow failureevents that require workflows to be reconfigured (e.g., drug in conflictwith patient allergy). This part of a clinical workflow managementsystem is, in other contexts, also referred to as a database ofmachine-readable clinical guidelines. Furthermore, a resources databasefor clinical equipment and clinical personnel (including description ofroles and responsibilities for each person) is used. Documentation ofresults from clinical actions, which may include e.g. diagnosticresults, diagnostic images, therapeutic decisions, or financial data onincurred cost, reimbursement IDs etc., is generally done in databasessuch as Hospital Information Systems (HIS), Radiology InformationSystems (RIS), and Electronic Patient Record (EPR). Clinical workflowmanagement permits intentional and optimized workflow process designs,and enforces the execution of these designs. They track the progress ofindividual work steps, and can automatically escalate failed or expiredwork steps.

For planning the sequence of action steps of a clinical task, andscheduling of all of these steps (including assignment of equipment usedand responsible persons), the workflow engine receives a request for anaction on a certain patient, for example (such as MR diagnosis of brainfor patient with patient ID XXX, for example). The workflow engineaccesses the definitions database to identify the action sequence to beplanned for this request and how it must fit with other existing orplanned action sequences, and accesses the resources database in orderto assign and schedule equipment and personal. The rules engine accessesthe rules for how specific actions must be modified based on patient andenvironmental factors. Workflow control is done e.g. by supervising thatrequired output from an action step has been entered into the system,before a subsequent action or a decision point is triggered, whichdepends on the output of that preceding action. Action steps performedand not performed can be saved historically for subsequent processmining and feedback to adjust and further optimize established actionsequences. Thus, in summary, the main parts of a Clinical WorkflowManagement System are a workflow engine, a rules Engine, and a databaseon clinical resources.

In the context of optimizing the quality of performance of a clinicalstudy through the use of a Clinical Workflow Management System,essentially requests for clinical actions or requirements constraints onclinical actions for a patient enrolled in a clinical study are takenfrom the SOP. From these SOP requirements, requests are generated,preferably in a machine-readable form, which are fed into the ClinicalWorkflow Management System. From this input, the Clinical WorkflowManagement System plans and schedules clinical actions, or respectivelymodifies already planned and scheduled clinical actions such that theyare compliant with the constraints defined by the SOP.

In the context of embodiments of the invention, the task of extractingrequirements for clinical actions from the SOP and feeding them into theClinical Workflow Management System may either be done manually by aperson who represents the role of the Hospital Information ServiceProvider (HISP) in the business and workflow model described, or it maypreferably be achieved automatically by software embedded in theanalytical device 200, which analyzes the SOP, extracts requirements,and feeds them into the Clinical Workflow Management System.

Accordingly, as shown in FIG. 3, in this prospective aspect of anembodiment of the present application, planning occurs in element 320;and can be done utilizing the clinical workflow management systemdatabase 210 and the analytical device 200, as shown in element 330, forexample. As such, a modified SOP-compliant clinical workflow may beestablished in element 340.

In the retrospective quality check and/or compliance measurement aspectof the present application, clinical workflow or other data can becompared against rules or other aspects/criteria of the clinical study,after the workflow has been executed or the data been measured anddocumented. This can also include involvement of a clinical workflowsystem and clinical data management system 210 as shown in element 335,and the report of clinical workflow which violates the SOP rules orother criteria for the clinical study can be established as shown inelement 345. As such, a method of checking on quality, and/ordetermining a measure of clinical study compliance can include obtainingcriteria for the clinical study from either SOP database 280 and/orcreated SOP Q-Base 300. Stored clinical data from databases 210, 212,and/or 214 for example, can be accessed if related to the clinicalstudy, or at least databases can be accessed for patients enrolled in aparticular clinical study for example. The criteria of clinical data canthen be correlated to determine a measure of compliance or measure ofquality regarding the criteria of the clinical study. Further, theclinical data may be stored in association with identificationinformation of patients enrolled in the clinical study, wherein onlyclinical data relating to patients enrolled in the clinical study arecorrelated with the criteria.

Further, the retrospective quality check aspect of the presentapplication can be used in a method of monitoring the clinical study ofthe clinical trial site, wherein at least one protocol for the clinicalstudy can be obtained from at least one of databases 280 and 300 forexample; clinical data relating to actions performed at the clinicaltrial site can be accessed from clinical workflow management system 210for example, based upon the at least one protocol; and the at least oneprotocol and the clinical data can be correlated, wherein the correlatedinformation can be used to monitor the clinical study at the trial site.

Thus, the retrospective quality check of the present application cancheck all reported clinical actions of clinical data, which belonged toa patient ID of a clinical study participant for example, bycross-checking these clinical actions and/or clinical data againstprotocol, criteria, etc. from the SOP database 280 and/or rules forclinical workflow established by the SOP Q-Base 300. In the case that athreshold of quality is not met, and/or a threshold measure ofcompliance is not reached for example, such a problem can beautomatically reported to the sponsor 220 for example. This can be doneby, for example, the sending of an electronic message. Alternatively,this information can be accumulated and reported at a later date.Optionally, a responsible person at the investigator's site or clinicaltrial site may be notified, such that an action can be corrected and thequality and/or compliance can be maintained.

It is important to note that normally, clinical action informationavailable on the clinical workflow management system 210 is notdocumented for a long period of time. For example, it is not normallydocumented as part of the electronic patient record (EPR) and stored inEPR 212. Rather, much of this information is volatile and used only inthe system until a clinical action is concluded. Therefore, thecross-check or correlation of clinical actions and/or clinical data withprotocol and/or criteria of a clinical study are done in a fairlyautomated and substantially real-time manor during execution of theclinical workflow, “substantially real-time” meaning that thecorrelation is done at some point before the volatile data are deleted.Alternatively, and/or additionally, workflow information of a patientenrolled in the clinical study may be stored on a long-term basis in aseparate file, for later off-line evaluation against clinical studycriteria for example. Thus, when an attempt is made to correlatevolatile clinical data with criteria for the clinical study and/orparameters of the clinical study, the volatile data may be permanentlystored upon determining a lack of correlation or measure compliancebelow a particular threshold for example. Further, volatile clinicaldata may be correlated with criteria for the clinical study, wherein thevolatile data are thereafter permanently stored upon determining thatthe measure of compliance is below a threshold.

In either aspect of a prospective embodiment of the present applicationor a retrospective quality check of the present application, a log fileof quality and/or measures of compliance with protocol and/or criteriaof the clinical study may be developed as shown in element 350. This logfile of quality checks and/or quality related clinical action planningor clinical action modification may then be delivered to the sponsor 220of the clinical study. In addition, and/or alternatively, based on sucha log-file, a quality and/or compliance score may be calculated inelement 360.

Thus, a method of analyzing quality of a clinical study can includecreating a database including quality related aspects from criteria forthe clinical study, and accessing stored clinical data relating to theclinical study. Thereafter, the method can include correlating thequality related aspects of criteria and clinical data to determine aquality and/or compliance score. The score may then be used to activatequality based studying centers as shown in element 370 for example, independence of the achieved quality and/or compliance; and/or can be usedfor ranking investigators and/or clinical trial or investigator sitesaccording to their quality and/or compliance scores as shown by element380. This may be especially meaningful if the quality/compliance ofseveral investigators and/or investigator sites is to be compared, allusing the same or comparable clinical workflow systems. The log-fileand/or the derived score may be communicated to the sponsor 100, forexample in repeated intervals throughout the clinical study period, fordetection of any quality-related changes.

The SOP regulation or other criteria and/or protocol of the study fromdatabase 280 can be translated automatically into the “language” thatthe analytical device 200 or clinical workflow system understandsutilizing an ontology, thesaurus, etc. Alternatively, if this is notpossible and must be solved interactively, then the necessary workloadcan be reduced by building a database with typical SOP rules. Such rulescan include but are not limited to, for example, taking a bloodsample/urine sample at a pre-determined time (in the morning, fasting,etc.) and handling it according to defined specifications; applyingmedication (in dose for example) at specific times corresponding to thepatient's status, which is interconnected to an understood clinicalworkflow systems or analytical device 200; etc. The database may includea GUI with a checklist, for example, wherein the user can check a boxfor each rule, relevant for the current SOP Q-Base SOP Q-Base 300 to beimplemented. This checklist will automatically grow with each newimplemented SOP, by adding new rules to the list as needed.

In another embodiment, a method can be performed for analyzing qualityof a clinical study. Such a method includes creating a databaseincluding quality related aspects from criteria for the clinical study,such as the SOP Q-base, for example. Thereafter, the analytical device200 for example, may access stored clinical data relating to theclinical study. Finally, the quality related aspects of criteria andclinical data may be correlated to determine performance of the qualityrelated aspects of the criteria of the clinical study. Such a method canfurther include, for example, calculating a measure of performance for aplurality of criteria for the clinical study.

As stated previously, analytical device 200 may access, for example,IT-based clinical workflow systems and clinical data management systemsfor automated control of quality regulations of clinical studies. Adatabase of machine-readable rules, defining quality related regulationsfrom criteria/protocol of the clinical study, may be derived from thestandard operating procedure document (SOP) of the clinical study 280.Such a database, which can include SOP Q-Base 300 for example, may thenbe automatically interrogated by the analytical device of the clinicalworkflow system 200, and deviation from procedures or other aspects ofnon-compliance may be documented, wherein the rules may automatically beused to modify the clinical workflow according to the requirements ofthe clinical study. This can enable, for example, objective, quantifiedand automatic measurement of documentation of quality/compliance inclinical studies; quality-oriented payment of study incentive; rankingof investigator/clinical trial site according to theirquality/compliance, etc.

In one embodiment, a first electronic database is built by theanalytical device 200. This database, which can include SOP Q-base 300for example, may be built from criteria for the clinical study (rules,values, thresholds, etc.) either automatically or manually via a GUI,extracting this information from clinical study criteria and/or protocolof database the clinical study SOP 280, for example.

The criteria can include various target quality measures, whereinclinical data of a plurality of clinical trial sites and the obtainedcriteria for the clinical study may be further analyzed to determineclinical trial sites which may exceed target quality measures of theobtained criteria. From various actual quality measures obtained, aranking of the quality measures can be derived, wherein the ranking isfor at least one of clinical trial sites, payment amounts and studydiscontinuation decisions (namely, decisions as to whether or not aclinical study should be discontinued). In addition, the determinedmeasure of compliance, as discussed in any of the aforementionedembodiments, may be used in at least one of : calculating a paymentbased upon the determined measure of compliance; calculating a rankingof compliance for a plurality of clinical trial sites, and at least oneof detecting and reporting quality compliance problems during a clinicalstudy.

Thus, the analytical device 200 may suggest discontinuation of the studyif one or more of the quality measures, rules, values and thresholds isnot met. This discontinuation of a clinical study at a particularclinical trial site, for example, may result in a large reduction inlosses or costs which may have been incurred if the study had beencontinued and unfavorable results were achieved. Thus, this can be alarge cost savings to the sponsor 220 and thus the analytical device 200may receive a contracted percentage of money not spent on a probablyunsuccessful study.

Again, in this embodiment, the criteria can include study protocol ofthe clinical study, wherein the clinical data may be obtained from atleast one electronic healthcare database including at least one of thosepreviously set forth. The criteria can include at least one of rules,values and thresholds.

Each of the various embodiments discussed above can include the use ofweighting factors. For example, the clinical study criteria obtained caninclude weighing factors, wherein the weighting factors may reflect alikelihood of the “criteria” to correlate with direct benefit, such asfinancial benefit, for example. Thus, weighted criteria may be used.This weighted criteria can include weighting that affects the measure ofcompliance, for example. With regard to protocol/parameters of clinicalstudy criteria such as study duration, costs, study result reliability,major “criteria” which may help to influence these measures positivelymay include, but are not limited to e.g.:

-   -   Overall number of patients which can be controlled in the study,        respectively number of patients per time unit which can be        enrolled;    -   Time-effectiveness of data collection and evaluation;    -   Compliance of investigator and patient with the study rules;    -   Experience/capability of the investigator to motivate patients        for continued participation until the end of the study, and not        drop out earlier;    -   Claimed amount of compensation from investigator and patient,        etc.

Often, this “criteria” cannot be measured directly, but must be deducedfrom other measurable parameters, and perhaps from a combination ofother measurable parameters. Thus, the analytical device 200 may, forexample, build an empirical database on typical “dropout” rates ofpatients, wherein these rates might vary with investigation sites,patient's age, geography, etc. Thus, the analytical device 200 cancreate a type of mathematical formula or weighting factors regarding thecombining of several direct and indirect criteria. Most likely, thisformula will include a weighted sum or weighted product of the singlecriteria. Accordingly, an output ranking may be derived and the rankingmay be based upon weighted determinations using the weighing factors.

Throughout various aspects and embodiments of the present application,correlation of criteria and/or protocol of a clinical study withclinical data (which can include patient data, reported clinicalactions, etc.) is discussed. One exemplary aspect of how suchcorrelation may be achieved, noting that the embodiments are not limitedas such, is as follows.

The clinical study criteria includes protocols which may contain, forexample, rules and/or constraints regarding the clinical studyprocedures. These can include, but are not limited to, e.g.:

-   -   do blood pressure check at least twice daily, e.g. in the        morning and in the evening;    -   do not subscribe medication X to clinical study patients;    -   when patient's blood pressure exceeds value Y, reduce dose of        medication Z by 50%;    -   patient's diet must not exceed 2000 Kcal per day.

Correlating the stored clinical study criteria (e.g. these rules andconstraints) with clinical data refers to, for example, comparing theserules and constraints (which are stored preferably in a machine-readableform) with clinical data including, but not limited to data entries inthe Electronic Patient Record (EPR) 212; input and output parameters forclinical actions in clinical workflow management system 210 for example;etc. From such a comparison, it can then be determined from these EPRworkflow management and/or other clinical data, whether or not therules, for example, of the protocol/criteria of the clinical study havebeen met. In simple cases, the “measurement for correlation” may be e.g.a 1 if the rule is met, and a zero, if the rule is not met. Thecompliance with some rules may also be measured in a more quantitativeway, e.g. for the rule “patients diet must not exceed 2000 Kcal/per day”can be measured with 100% for all diets less or equal to 2000 Kcal/perday, and being 90% for 2200 Kcal, 50% for 3000 Kcal etc. Thus, a measureof compliance may be determined. Further, the correlated information canbe used to monitor the clinical study at the clinical trial site; and/orat a plurality of clinical trial sites.

Compliance, as used throughout the embodiments of the presentapplication, refers to participants of the clinical study (investigator,patient, clinical trial site, etc.) are following the rules andconstraints defined in the study protocol/criteria of the clinicalstudy. The most important example for compliance is certainty, e.g. thata medication under test has to be taken by the patient exactly in theprescribed manner (number of pills and times of day when the medicationhas to be taken, before or after meals, taken together with a minimumamount of liquid, etc.). The embodiments of the present applicationinclude both patient compliance (and/or patient non-compliance) andinvestigator/clinical trial site compliance to follow the protocolrules; and further include analysis of the skills and ability of theinvestigator to perform certain tasks, and the suitability of clinicalprocesses in the context of these protocol rules. For example, aphysician with limited experience in Ultrasound exams may well becompliant with all rules to the best of his knowledge, but still maydeliver inferior results as compared to a very experienced colleague.For these reasons, compliance also includes “quality criteria”, “qualityimprovement”, and/or “quality assurance”.

One non-limiting example of the methodology of an embodiment of thepresent application is as follows. Again, this is a non-limitingIllustrating example.

In this non-limiting example, a patient in a hospital is enrolled in aclinical study. The protocol of the clinical study, obtained as criteriaof the clinical study includes:

-   -   A) the patient's blood pressure is measured at least 3 times        daily, between 6-8 am, 12-2 pm, and 18-20 pm respectively, and    -   B) that an ultrasound exam is performed once weekly, with the        patient being in a fasting condition

The clinical data, which may be accessed using a patient ID of theparticipating patient for example, is correlated with the relevantclinical protocol, e.g. by entering a reference to the clinical protocolrules database (SOP Q-Base 300) in the patient's EPR 212. Then the SOPQ-Base 300 is analyzed by the analytical device 200 for applicablerules. When rule (A) is identified, the clinical action “blood pressuremeasurement” is scheduled or planned at least partially automaticallyeach day according to the rules, and appropriate human resourcesprovided (in the prospective planning aspect).

The results of the blood pressure measurement, together with time anddate of the exam, are then entered into the patient's EPR 212. This datamay then be analyzed periodically and extracted into the patients studylog-file, for example. In case that measurement values are missing, orare not taken within the defined time frame, this fact may be loggedand/or automatically reported e.g. to the sponsor 220 (e.g. if thedeviations exceed a predefined threshold).

When rule (B) is encountered, weekly ultrasound exams are scheduled orplanned for the patient, at times preceding the scheduled breakfast,such that the patients is in a fasting condition. The rule is enteredinto the patient's EPR 212, such that any possible rescheduling of aspecific ultrasound exam is preferably only done in combination withaccording scheduling of the breakfast on this day. Times of actuallycarried-out exams may be recorded in the EPR 212, optionally togetherwith the delivery time of the breakfast, such that retrospective checkof rule compliance may be carried out in addition to this prospectiveclinical workflow planning.

Number and severity of deviations from these rules may thenautomatically calculated from the entries in the patient's EPR 212, forexample. For each rule, a compliance measure may then be calculated fromthe sum of all deviations from this rule for example. Alternatively, oroptionally, for all rules together a combined measure is calculated toquantify the overall quality of this patient's participation in thestudy. If the individual or combined quality measures are exceedingpredefined thresholds, predefined actions may automatically be taken,such as excluding the patient from the study, excluding the clinicaltrial site from the study, reducing the payment for the clinical trialsite negotiating action plans how to improve the performance of thetrial site, etc.

A method of improving compliance within at least one clinical study at aclinical trial site can thus include obtaining at least one protocol forthe clinical study, and planning a plurality of clinical actions incompliance with criteria set forth in the at least one protocol. Theplanning may be performed at least partially automatically. wherein thecriteria includes at least one of rules and constraints. At least one ofrules and constraints may be stored in machine readable form. Further,the planning may include at least one of scheduling use of equipment atthe clinical trial site and/or at least one of scheduling use ofpersonnel at the clinical trial site. Additionally, the clinical trialsite may be a hospital.

Any of the aforementioned methods may be embodied in the form of asystem or device, including, but not limited to, any of the structurefor performing the methodology illustrated in the drawings. Suchstructure may include, for example, a database, storing criteria for theclinical study. Further, it may include one or more devices foraccessing stored clinical data relating to the clinical study, and forcorrelating the criteria and clinical data to determine a measure ofcompliance with the criteria of the clinical study. In anotherembodiment, the system may be for monitoring a clinical study at aclinical trial site, and may include one or more devices for obtainingat least one protocol for the clinical study; for accessing storedclinical data relating to actions performed at the clinical trial sitebased upon the at least one protocol; and for correlating the at leastone protocol and clinical data and using the correlated information tomonitor the clinical study at the clinical trial site. In yet anotherembodiment, the system may be for improving compliance within at leastone clinical study at a clinical trial site and may include one or moredevices for obtaining at least one protocol for the clinical study; andfor planning a plurality of clinical actions in compliance with criteriaset forth in the at least one protocol, wherein the planning isperformed at least partially automatically. Further, in still anotherembodiment, the system may be for creating a database including qualityrelated aspects from criteria for the clinical study. The system mayinclude one or more devices for accessing stored clinical data relatingto the clinical study; and for correlating the quality related aspectsof criteria and clinical data to determine performance of the qualityrelated aspects of the criteria of the clinical study.

Further, any of the aforementioned methods may be embodied in the formof a program. The program may be stored on a computer readable media andis adapted to perform any one of the aforementioned methods when run ona computer device (a device including a processor). Thus, the storagemedium or computer readable medium, is adapted to store information andis adapted to interact with a data processing facility or computerdevice to perform the method of any of the above mentioned embodiments.

The storage medium may be a built-in medium installed inside a computerdevice main body or removable medium arranged so that it can beseparated from the computer device main body. Examples of the built-inmedium include, but are not limited to, rewriteable involatile memories,such as ROMs and flash memories, and hard disks. Examples of theremovable medium include, but are not limited to, optical storage mediasuch as CD-ROMs and DVDs; magneto-optical storage media, such as MOs;magnetism storage media, such as floppy disks (trademark), cassettetapes, and removable hard disks; media with a built-in rewriteableinvolatile memory, such as memory cards; and media with a built-in ROM,such as ROM cassettes.

Exemplary embodiments being thus described, it will be obvious that thesame may be varied in many ways. Such variations are not to be regardedas a departure from the spirit and scope of the present invention, andall such modifications as would be obvious to one skilled in the art areintended to be included within the scope of the following claims.

1-141. (canceled)
 142. A method of analyzing quality of a clinicalstudy, comprising: creating a database including quality related aspectsfrom criteria for the clinical study; accessing stored clinical datarelating to the clinical study; and correlating the quality relatedaspects of criteria and clinical data to determine performance of thequality related aspects of the criteria of the clinical study.
 143. Themethod of claim 142, further comprising: calculating a measure ofperformance for a plurality of criteria for the clinical study.
 144. Themethod of claim 142, wherein the clinical data includes clinicalactions.
 145. The method of claim 143, wherein the measure ofperformance includes patient performance.
 146. The method of claim 143,wherein the measure of performance includes performance of a clinicaltrial site wherein the clinical trial is performed.
 147. The method ofclaim 142, wherein the correlating is done via a computer device. 148.The method of claim 142, wherein the clinical data and criteria for theclinical data are stored in separate databases.
 149. The method of claim142, wherein the stored criteria includes quality aspects related topatient performance criteria and quality aspects related to non-patientperformance criteria.
 150. The method of claim 143, wherein the criteriaincludes weighted criteria, wherein the weighting affects the measure ofperformance.
 151. The method of claim 142, wherein the criteria includemeasurable parameters and parameters of context.
 152. The method ofclaim 150, wherein the criteria include measurable parameters andparameters of context.
 153. A device for implementing the method ofclaim
 142. 154. A program, adapted to perform the method of claim 142,when executed on a computer.
 155. A computer readable medium, storingthe program of claim
 154. 156. The method of claim 142, wherein theclinical data is stored in association with identification informationof patients enrolled in the clinical study, wherein only clinical datarelating to patients enrolled in the clinical study are correlated withthe criteria.
 157. The method of claim 156, wherein the clinical dataand identification information of patients enrolled in the clinicalstudy are stored in a single database.
 158. The method of claim 156,wherein the clinical data and identification information of patientsenrolled in the clinical study are stored in separate databases. 159.The method of claim 142, further comprising: informing the clinicaltrial site if the quality related aspects of criteria and clinical datado not correlate.
 160. The method of claim 143, further comprising:informing the clinical trial site if the measure of performance is belowa threshold.
 161. The method of claim 159, wherein the informing isachieved via an electronic message.
 162. The method of claim 160,wherein the informing is achieved via an electronic message.
 163. Themethod of claim 142, wherein an attempt is made to correlate volatileclinical data with criteria for the clinical study, and wherein thevolatile data are permanently stored upon determining a lack ofcorrelation.
 164. The method of claim 143, wherein volatile clinicaldata are correlated with criteria for the clinical study, and whereinthe volatile data are permanently stored upon determining that themeasure of performance is below a threshold.
 165. The method of claim142, wherein clinical data are stored permanently, upon determining thatthe data relate to a patient enrolled in the clinical study.
 166. Themethod of claim 156, wherein clinical data are stored permanently, upondetermining that the data relate to a patient enrolled in the clinicalstudy.
 167. The method of claim 142, wherein a listing of correlatedclinical data is reported to a sponsor of the clinical study.
 168. Themethod of claim 143, wherein the measure of performance is reported to asponsor of the clinical study.
 169. The method of claim 142, wherein thecriteria for the clinical study are automatically translated into ruleswhich are machine-readable.
 170. The method of claim 169, wherein therules are machine-readable via at least one of a look-up table, athesaurus, and an ontology.
 171. The method of claim 142, wherein thecriteria for the clinical study are translated into rules which aremachine-readable and wherein applicable criteria are selected from adatabase.
 172. A system for analyzing quality of a clinical study,comprising: means for creating a database including quality relatedaspects from criteria for the clinical study; means for accessing storedclinical data relating to the clinical study; and means for correlatingthe quality related aspects of criteria and clinical data to determineperformance of the quality related aspects of the criteria of theclinical study.
 173. The system of claim 172, further comprising: meansfor calculating a measure of performance for a plurality of criteria forthe clinical study.
 174. The system of claim 172, wherein the clinicaldata includes clinical actions.
 175. The system of claim 173, whereinthe measure of performance includes patient performance.
 176. The systemof claim 173, wherein the measure of performance includes performance ofa clinical trial site wherein the clinical trial is performed.
 177. Thesystem of claim 172, wherein the correlating is done using a computerdevice.
 178. The system of claim 172, wherein the clinical data andcriteria for the clinical data are stored in separate databases. 179.The system of claim 172, wherein the stored criteria includes qualityaspects related to patient performance criteria and quality aspectsrelated to non-patient performance criteria.
 180. The system of claim173, wherein the criteria includes weighted criteria, wherein theweighting affects the measure of performance.
 181. The system of claim172, wherein the criteria include measurable parameters and parametersof context.
 182. The system of claim 180, wherein the criteria includemeasurable parameters and parameters of context.
 183. The system ofclaim 172, wherein the clinical data is stored in association withidentification information of patients enrolled in the clinical study,wherein only clinical data relating to patients enrolled in the clinicalstudy are correlated with the criteria.
 184. The system of claim 183,wherein the clinical data and identification information of patientsenrolled in the clinical study are stored in a single database.
 185. Thesystem of claim 183, wherein the clinical data and identificationinformation of patients enrolled in the clinical study are stored inseparate databases.
 186. The system of claim 172, further comprising:means for informing the clinical trial site if the quality relatedaspects of criteria and clinical data do not correlate.
 187. The systemof claim 173, further comprising: means for informing the clinical trialsite if the measure of performance is below a threshold.
 188. The systemof claim 186, wherein the informing is achieved via an electronicmessage.
 189. The system of claim 187, wherein the informing is achievedvia an electronic message.
 190. The system of claim 172, wherein anattempt is made to correlate volatile clinical data with criteria forthe clinical study, and wherein the volatile data are permanently storedupon determining a lack of correlation.
 191. The system of claim 173,wherein volatile clinical data are correlated with criteria for theclinical study, and wherein the volatile data are permanently storedupon determining that the measure of performance is below a threshold.192. The system of claim 172, wherein clinical data are storedpermanently, upon determining that the data relate to a patient enrolledin the clinical study.
 193. The system of claim 183, wherein clinical,data are stored permanently, upon determining that the data relate to apatient enrolled in the clinical study.
 194. The system of claim 172,wherein a listing of correlated clinical data is reported to a sponsorof the clinical study.
 195. The system of claim 173, wherein the measureof performance is reported to a sponsor of the clinical study.
 196. Thesystem of claim 172, wherein the criteria for the clinical study areautomatically translated into rules which are machine-readable.
 197. Thesystem of claim 196, wherein the rules are machine-readable via at leastone of a look-up table, a thesaurus, and an ontology.
 198. The system ofclaim 172, wherein the criteria for the clinical study are translatedinto rules which are machine-readable and wherein applicable criteriaare selected from a database. 199-213. (canceled)
 214. A method ofanalyzing quality of a clinical study, comprising: creating a databaseincluding quality related aspects from criteria for the clinical study;accessing stored clinical data relating to the clinical study; andcorrelating the quality related aspects of criteria and clinical data todetermine at least one of a quality score and a compliance score.
 215. Adevice for implementing the method of claim
 214. 216. A program, adaptedto perform the method of claim 214, when executed on a computer.
 217. Acomputer readable medium, storing the program of claim 215.